Intelligent Construction Case Study Illustration System Using Natural Language Processing and Image Searching

With the rapid advancement in science and engineering disciplines, the amount of knowledge exploded. Therefore, it is more and more difficult for a single person to become an expert in multiple domains. In this regard, an intelligent illustration system is beneficial to help people quickly understand textual description of case studies in different domains. This is especially the case in construction engineering and management education, where the students may come from different backgrounds (e.g., architecture, civil engineering and non-civil related domains). And the students may not understand many terminologies used in construction projects (e.g., toenailing, sandblasting and Canadian snow load) which could hinder them from adequately analyzing construction case studies. To address this challenge, this paper proposes a new intelligent illustration system which takes a textual description of a construction case study and renders a set of structured images with each image illustrating an important concept in the textual description. The system is based on natural language processing and keywords-based image searching techniques. Part-of-speech (POS) tagging and dependency parsing are used to process a sentence to generate POS tags for each word, and dependency relations, respectively. Both POS tags and dependency relations are used in extracting the interesting concepts and relations from the case study. The concepts are then fed into the image searching process as the keywords. The retrieved images are organized into structured images to illustrate the interesting concepts, where the image result for each concept is connected to another image result using the corresponding relation. A preliminary implementation of the system was tested on a construction management case study. All interesting concepts and relations were successfully extracted and automatically assigned result images for illustrations. It was observed that some result images were not quite relevant to the input case study. As such, this will be addressed in future work by adding contextual terms to image search keywords.